Recent progress in natural language processing has been driven by advances in both model architecture and model pretraining. Transformer architectures have facilitated building higher-capacity models and pretraining has made it possible to effectively utilize this capacity for a wide variety of tasks. Transformers is an open-source library with the goal of opening up these advances to the wider machine learning community. The library consists of carefully engineered stateof-the art Transformer architectures under a unified API. Backing this library is a curated collection of pretrained models made by and available for the community. Transformers is designed to be extensible by researchers, simple for practitioners, and fast and robust in industrial deployments. The library is available at https://github.com/ huggingface/transformers.

Adversarial attacks hamper the decision-making ability of neural networks by perturbing the input signal. The addition of calculated small distortion to images, for instance, can deceive a well-trained image classification network. In this work, we propose a novel attack technique called Sparse Adversarial and Interpretable Attack Framework (SAIF). Specifically, we design imperceptible attacks that contain low-magnitude perturbations at a small number of pixels and leverage these sparse attacks to reveal the vulnerability of classifiers. We use the Frank-Wolfe (conditional gradient) algorithm to simultaneously optimize the attack perturbations for bounded magnitude and sparsity with $O(1/\sqrt{T})$ convergence. Empirical results show that SAIF computes highly imperceptible and interpretable adversarial examples, and outperforms state-of-the-art sparse attack methods on the ImageNet dataset.

Artificial intelligence (AI) has enormous potential to improve Air Force pilot training by providing actionable feedback to pilot trainees on the quality of their maneuvers and enabling instructor-less flying familiarization for early-stage trainees in low-cost simulators. Historically, AI challenges consisting of data, problem descriptions, and example code have been critical to fueling AI breakthroughs. The Department of the Air Force-Massachusetts Institute of Technology AI Accelerator (DAF-MIT AI Accelerator) developed such an AI challenge using real-world Air Force flight simulator data. The Maneuver ID challenge assembled thousands of virtual reality simulator flight recordings collected by actual Air Force student pilots at Pilot Training Next (PTN). This dataset has been publicly released at Maneuver-ID.mit.edu and represents the first of its kind public release of USAF flight training data. Using this dataset, we have applied a variety of AI methods to separate "good" vs "bad" simulator data and categorize and characterize maneuvers. These data, algorithms, and software are being released as baselines of model performance for others to build upon to enable the AI ecosystem for flight simulator training.

Our education system comprises a series of curricula. For example, when we learn mathematics at school, we learn in order from addition, to multiplication, and later to integration. Delineating a curriculum for teaching either a human or a machine shares the underlying goal of maximizing the positive knowledge transfer from early to later tasks and minimizing forgetting of the early tasks. Here, we exhaustively surveyed the effect of curricula on existing continual learning algorithms in the class-incremental setting, where algorithms must learn classes one at a time from a continuous stream of data. We observed that across a breadth of possible class orders (curricula), curricula influence the retention of information and that this effect is not just a product of stochasticity. Further, as a primary effort toward automated curriculum design, we proposed a method capable of designing and ranking effective curricula based on inter-class feature similarities. We compared the predicted curricula against empirically determined effectual curricula and observed significant overlaps between the two. To support the study of a curriculum designer, we conducted a series of human psychophysics experiments and contributed a new Continual Learning benchmark in object recognition. We assessed the degree of agreement in effective curricula between humans and machines. Surprisingly, our curriculum designer successfully predicts an optimal set of curricula that is effective for human learning. There are many considerations in curriculum design, such as timely student feedback and learning with multiple modalities. Our study is the first attempt to set a standard framework for the community to tackle the problem of teaching humans and machines to learn to learn continuously.

目的：本研究评估了市售可解释的AI算法在增强临床医生在胸部X射线（CXR）上鉴定肺癌的能力的影响。设计：这项回顾性研究评估了11位临床医生在胸部X光片中检测肺癌的表现，并在有和没有市售的AI算法的帮助下（红点，观察到），预测CXRS可疑的肺癌。根据临床确定的诊断评估了临床医生的表现。设置：该研究分析了NHS医院的匿名患者数据；该数据集由成年患者（18岁及以上）的400张胸部X光片组成，他们在2020年进行了CXR，并提供相应的临床文本报告。参与者：由11位临床医生（放射科医生，放射科医生受训者和报告射线照相师）组成的读者小组参加。主要结果指标：临床医生在CXR上检测肺癌的总体准确性，敏感性，特异性和精度，有或没有AI输入。还评估了有或没有AI输入的临床医生与绩效标准偏差之间的协议率。结果：临床医生对AI算法的使用导致肺部肿瘤检测的总体性能提高，从而达到了在CXR上鉴定出的肺癌的总体增长17.4％ ，分别增加了13％和13％的阶段1和2期肺癌的检测，以及临床医生表现的标准化。结论：这项研究在AI算法的临床实用性方面表现出了巨大的希望，可以通过整体改善读者表现来改善早期肺癌诊断和促进健康平等，而不会影响下游成像资源。

我们探索不同的策略，将先前的领域知识整合到深神经网络（DNN）的设计中。我们专注于图形神经网络（GNN），其用例是估计表示为图的化学系统（分子和晶体）的势能。我们将域知识的两个要素集成到GNN的设计中，以限制和正规化其学习，以提高准确性和泛化。首先，关于原子之间不同类型关系（化学键）存在的知识用于调节GNN中的节点的相互作用。其次，对某些物理数量的相关性的知识用于使用简单的多任务范式将学习的特征限制为更高的物理相关性。我们通过将它们应用于两个依赖不同机制来传播节点和更新节点状态的不同机制的架构来证明我们的知识集成的一般适用性。

当人类共同完成联合任务时，每个人都会建立一个情况的内部模型以及如何发展。有效的协作取决于这些单个模型如何重叠以在团队成员之间形成共同的心理模型，这对于人类机器人团队中的协作流程很重要。准确的共享心理模型的发展和维护需要个人意图的双向交流以及解释其他团队成员意图的能力。为了实现有效的人类机器人协作，本文介绍了人类机器人团队合作中新型联合行动框架的设计和实施，利用增强现实（AR）技术和用户眼目光来实现意图的双向交流。我们通过与37名参与者的用户研究测试了我们的新框架，发现我们的系统提高了任务效率，信任和任务流利。因此，使用AR和眼睛凝视使双向交流是一种有前途的平均值，可以改善影响人与机器人之间协作的核心组成部分。

随着基于人工智能（AI）和机器学习（ML）技术的实用性的增长，对抗性攻击的威胁越来越大。有必要将这个生态系统的团队红色团结起来，以确定系统漏洞，潜在威胁，表征将增强系统鲁棒性并鼓励创造有效防御的属性。次要的需求是在不同的利益相关者，模型开发人员，用户和AI/ML安全专业人员等不同的利益相关者之间分享此AI安全威胁情报。在本文中，我们创建并描述了原型系统CTI4AI，以克服有条不紊地识别和共享AI/ML特定漏洞和威胁智能的需求。

数据增强是改善深神经网络概括的必不可少的技术。大多数现有的图像域增强剂要么依赖几何和结构变换，要么应用不同种类的光度扭曲。在本文中，我们提出了一种有效的技术，可以通过将上下文有意义的知识注入场景中。我们通过语言接地（Semaug）进行对象检测的语义上有意义的图像增强方法首先计算出可以将其放置在图像中相关位置的语义上适当的新对象（问题和位置）。然后，它将这些对象嵌入其相关目标位置，从而促进对象实例分布的多样性。我们的方法允许介绍培训集中可能不存在的新对象实例和类别。此外，它不需要培训上下文网络的额外开销，因此可以轻松地将其添加到现有架构中。我们全面的评估集表明，所提出的方法在改善概括方面非常有效，而开销可以忽略不计。特别是，对于广泛的模型体系结构，我们的方法分别在Pascal VOC和COCO数据集上实现了约2-4％和〜1-2％的MAP改进。

通过一系列联邦举措和命令，美国政府一直在努力确保美国在AI中的领导。这些广泛的战略文件影响了美国空军美国部（DAF）等组织。DAF-MIT AI加速器是DAF和MIT之间的一项计划，以弥合AI研究人员与DAF任务要求之间的差距。DAF-MIT AI加速器支持的几个项目正在开发公共挑战问题，这些问题解决了许多联邦AI研究的重点。这些挑战是通过公开可用的大型AI-Ready数据集，激励开源解决方案，并为可以激发进一步研究的双重使用技术创建需求信号，来针对优先事项。在本文中，我们描述了正在开发的这些公共挑战以及它们的应用如何促进科学进步。